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EXECUTIVE  SUMMARY 


Weston  Solutions,  Inc.  performed  two  Environmental  Security  Technology  Certification 
Program  (ESTCP)  Live  Site  Demonstrations  of  advanced  geophysical  classification  (AGC) 
technologies  at  Southwestern  Proving  Ground  (SWPG),  Arkansas,  and  Andersen  Air  Force  Base 
(AFB),  Guam.  Detailed  infonnation  about  each  demonstration  is  contained  in  their  respective 
Final  Reports  published  on  the  ESTCP  website.  This  document  is  intended  to  summarize  the 
demonstrations  at  each  of  these  sites  and  assess  the  perfonnance  and  cost  benefits  of 
implementing  AGC  technologies. 

OBJECTIVES  OF  THE  DEMONSTRATION 

The  demonstrations  were  designed  to  validate  the  effectiveness  of  the  advanced  sensors  and 
classification  methodology  at  these  sites  containing  a  diversity  of  munitions,  densities,  and 
physical  challenges. 

The  primary  objectives  of  each  individual  Live  Site  Demonstration  were  to  do  the  following: 

•  Correctly  classify  all  targets  of  interest  (TOIs). 

•  Correctly  identify  TOI  and  non-TOI  sizes. 

•  Correctly  estimate  extrinsic  parameters  (measured  location  and  depths  of  items). 

•  Reduce  clutter  digs  by  at  least  50%. 

•  Extract  reliable  parameters  for  at  least  95%  of  cued  anomalies. 

•  Demonstrate  the  cost  benefit  and  perfonnance  of  these  sensors  and  methods. 

TECHNOLOGY  DESCRIPTION 

These  demonstrations  consisted  of  dynamic  and  cued  data  collection  with  the  MetalMapper  and 
TEMTADS  2x2  advanced  geophysical  sensor  systems.  Analysis  of  the  data  was  perfonned  using 
conventional  and  advanced  data  processing  methods  to  select  anomalies  from  the  advanced 
sensor  dynamic  detection  data,  and  then  extract  features  and  perform  anomaly  classification  on 
the  advanced  sensor  cued  data. 

DEMONSTRATION  RESULTS 

SWPG:  Weston  conducted  the  field  demonstration  in  three  phases  with  4-6  weeks  between  each 
phase  to  perform  data  processing  and  classification.  The  initial  phase  included  site  setup,  surface 
sweep,  production  area  seeding,  and  dynamic  data  collection  of  the  survey  area.  A  total  of  43 
seeds  were  installed,  and  1 1 .23  acres  of  dynamic  surveys  were  performed  with  the  MetalMapper 
system.  A  total  of  2,116  targets  were  selected  from  the  dynamic  data  for  cued  investigation, 
which  was  performed  during  the  second  phase.  Weston  returned  to  SWPG  for  the  final  intrusive 
phase,  during  which  1,398  targets  were  intrusively  investigated.  The  classification  methodology 
resulted  in  the  correct  classification  of  100%  of  TOI,  and  yielded  a  reduction  in  clutter  digs  of 
83%. 
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Andersen  AFB:  The  demonstration  was  integrated  with  an  ongoing  munitions  and  explosives  of 
concern  (MEC)  removal  action  project  being  performed  in  advance  of  military  construction 
(MILCON)  activities.  Dynamic  detection  surveys  using  TEMTADS  2x2  were  performed  across 
2.97  acres  of  the  site  following  a  traditional  EM61-MK2  Metal  Detector  survey.  A  total  of  970 
targets  were  selected  from  the  TEMTADS  2x2  dynamic  survey  data.  An  additional  225  targets 
were  selected  from  EM61-MK2  data  that  did  not  overlap  with  the  TEMTADS  2x2  survey.  Each 
of  the  1,195  anomalies  were  reacquired  and  interrogated  using  cued  data  collection  with  the 
TEMTADS  2x2.  All  TOI  were  correctly  identified  during  the  demonstration.  The  classification 
process  resulted  in  correctly  identifying  100%  of  TOI  and  reduced  the  number  of  clutter  or  non- 
munitions-related  material  that  would  require  investigation  by  81%. 

IMPLEMENTATION  ISSUES 

Continual  assessments  on  these  advanced  sensor  technologies  at  demonstration  sites  is  imperative 
to  the  continued  development  of  the  instruments.  This  report  will  discuss  implementation  issues 
related  to  equipment  shipping,  weatherproofing,  configurations,  structural  improvements,  and 
potential  modifications  observed  during  these  demonstrations.  These  implementation  issues  are 
documented  and  discussed  to  continually  assist  in  the  development  of  a  more  market-ready 
equipment,  and  to  facilitate  the  understanding  and  acceptance  of  advanced  geophysical  sensors  to 
improve  deployment  efficiency. 
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1.0  INTRODUCTION 


1.1  BACKGROUND 

Weston  Solutions,  Inc.  performed  an  Environmental  Security  Technology  Certification 
Program  (ESTCP)  Live  Sites  Demonstration  of  advanced  geophysical  classification  (AGC) 
sensor  technologies  at  two  project  locations:  Southwestern  Proving  Ground  (SWPG), 
Arkansas,  and  Andersen  Air  Force  Base  (AFB),  Guam.  Detailed  information  regarding  each 
site  location,  survey  design,  and  technology  results  is  contained  in  respective  Final  Reports 
published  on  the  Strategic  Environmental  Research  and  Development  Program  (SERDP)/ 
ESTCP  website. 

1.2  OBJECTIVE  OF  THE  DEMONSTRATION 

It  is  the  goal  of  ESTCP  to  utilize  AGC  to  increase  efficiency  of  the  Military  Munitions 
Response  Program  (MMRP)  and  reduce  costs  associated  with  intrusive  investigations  of 
harmless  metallic  objects  (e.g.,  range-related  debris,  or  cultural  debris).  The  demonstrations 
referred  to  in  this  report  were  designed  to  validate  the  effectiveness  of  the  advanced  sensors 
and  classification  methodology  at  these  sites  containing  a  diversity  of  munitions,  densities,  and 
physical  challenges. 

The  primary  objectives  of  each  individual  demonstration  of  AGC  activities  were  to  do  the 
following: 

•  Correctly  classify  all  targets  of  interest  (TOIs). 

•  Correctly  identify  TOI  and  non-TOI  sizes. 

•  Correctly  estimate  extrinsic  parameters  (measured  location  and  depths  of  items). 

•  Reduce  clutter  digs  by  at  least  50%. 

•  Extract  reliable  parameters  for  at  least  95%  of  cued  anomalies. 

•  Demonstrate  the  cost  benefits  and  performance  of  these  sensors  and  methods. 

1.3  REGULATORY  DRIVERS 

The  MMRP  is  charged  with  characterizing  and,  where  necessary,  remediating  Munitions 
Response  Sites  (MRSs).  When  an  MRS  is  remediated,  it  is  typically  mapped  with  a  geophysical 
system,  based  on  either  a  magnetometer  or  an  electromagnetic  induction  (EMI)  sensor,  and  the 
locations  of  all  detectable  signals  are  excavated.  Many  of  these  detections  do  not  correspond  to 
munitions,  but  rather  to  harmless  metallic  objects  or  geologic  features.  Field  experience  indicates 
that  often  in  excess  of  90%  of  objects  excavated  during  the  course  of  a  munitions  response  are 
found  to  be  nonhazardous  items.  Current  geophysical  technology,  as  it  is  traditionally 
implemented,  does  not  provide  a  physics-based,  quantitative,  validated  means  to  discriminate 
between  hazardous  munitions  and  nonhazardous  items. 
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With  no  information  to  suggest  the  origin  of  the  signals,  all  anomalies  are  currently  treated  as 
though  they  are  intact  munitions  when  they  are  dug.  They  are  carefully  excavated  by  unexploded 
ordnance  (UXO)  technicians  using  a  process  that  often  requires  expensive  safety  measures,  such 
as  barriers  or  exclusion  zones.  As  a  result,  most  of  the  costs  to  remediate  an  MRS  are  currently 
spent  on  excavating  targets  that  pose  no  threat.  If  these  items  could  be  detennined  with  high 
confidence  to  be  nonhazardous,  some  of  these  expensive  measures  could  be  eliminated  or  the 
items  could  be  left  unexcavated  entirely. 
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2.0  TECHNOLOGY 


AGC  technologies  and  methodologies  utilized  for  each  demonstration  is  summarized  in  Table  2.1. 


Table  2.1.  Classification  Technology  Used  at  the  Demonstration  Sites 


Demonstrated 

Technology 

Southwestern  Proving  Ground 
(SWPG) 

Andersen  AFB 

Geophysical  Survey 

MetalMapper  in  a  towed  configuration 
for  dynamic  and  cued  surveys. 

TEMTADS  2x2  in  a  person-portable 
cart  configuration  for  dynamic  and 
cued  surveys. 

Software  for  Data  Analysis 

Geosoft  Oasis  montaj  UX- Analyze  extension. 

Analysis  Methods 

Library  matching  and  cluster  analysis.  | 

Analysis  Categories 

Category  0:  Cannot  Analyze 

Category  1 :  Likely  TOI 

Category  2:  Cannot  Decide 

Category  3:  Likely  Non-TOI 

2.1  TECHNOLOGY  DESCRIPTION 

2.1.1  Geometries  MetalMapper 

The  Geometries  MetalMapper  is  the  first  commercially  available  advanced  EMI  sensor 
designed  to  enable  classification  of  TOI.  It  consists  of  three  orthogonal  1 -square-meter  (m2) 
transmit  coils  and  seven  10-centimeter  (cm),  three-component,  orthogonal  receiver  coils.  The 
system  was  proven  at  the  ESTCP  Live  Site  Demonstrations  at  the  former  Camp  San  Luis 
Obispo  and  other  live  sites  to  be  effective  at  discriminating  between  munitions  and  non¬ 
munitions  items.  Weston  operated  the  MetalMapper  in  both  dynamic  detection  and  cued 
interrogation  modes  during  the  Live  Site  Demonstration  at  SWPG.  The  MetalMapper  provides 
more  accurate  target  positioning  advantages  over  currently  used  technologies  (e.g.,  EM61- 
MK2)  because  of  its  seven  three-component  receivers,  greater  data  density,  and  improved 
positioning  electronics. 

2.1.2  TEMTADS  2X2 

The  TEMTADS  2x2  is  an  adaptation  of  the  Naval  Research  Lab’s  standard  TEMTADS  5x5 
element  sensor  configuration  using  a  smaller  2x2  element  array.  The  TEMTADS  2x2  consists  of 
four  35-cm  transmit  coils  with  four  8-cm  tri-axial  receiver  cubes.  The  receiver  cubes  are  similar 
in  design  to  those  used  in  the  second-generation  Advanced  Ordnance  Locator  and  the 
Geometries  MetalMapper  system  with  dimensions  of  8  cm  rather  than  10  cm.  It  is  as  reliable  as 
the  original  TEMTADS,  but  its  portability  and  smaller  size  enables  access  to  difficult  terrain 
where  mobility  is  limited.  The  center-to-center  distance  between  the  transmit  coils  is  40  cm, 
yielding  an  80  cm  x  80  cm  array.  The  array  is  deployed  on  a  set  of  wheels  resulting  in  a  sensor 
height  of  approximately  18  cm. 
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The  transmitter  electronics  and  the  data  acquisition  computer  are  mounted  on  the  operator 
backpack,  and  a  global  positioning  system  (GPS)  antenna  and  an  inertial  measurement  unit 
(IMU)  are  mounted  above  the  center  of  the  TEMTADS  2x2  sensor  array.  The  TEMTADS  2x2 
can  be  operated  in  two  modes:  dynamic  (or  detection)  mode  and  cued  mode.  Data  collection  is 
controlled  in  dynamic  mode  using  the  EM3DAcquire  application  suite,  similar  to  that  used  for 
the  Geometries  MetalMapper  systems.  Custom  software  written  by  the  Naval  Research 
Laboratory  (NRL)  is  used  for  cued  data  acquisition.  In  cued  mode,  the  locations  of  previously- 
identified  anomalies  are  reacquired  and  flagged  prior  to  being  cued  with  the  TEMTADS  2x2. 

2.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

2.2.1  MetalMapper 

The  MetalMapper  has  a  large  sensor  and  bulky  components  required  for  its  operation  (Microsoft 
Windows  computer,  monitor,  12-volt  batteries);  therefore,  a  deployment  vehicle  is  required 
during  surveys.  For  this  demonstration,  a  diesel-powered  tele-lifter  was  used  for  survey 
activities.  The  size  and  portability  of  the  MetalMapper  in  the  open  field  at  SWPG  became  an 
issue  because  of  clay-rich  soils  that  were  saturated  from  periodic  heavy  rains.  These  field 
conditions  impacted  all  phases  of  field  operations  and  likely  would  have  had  similar  impact  on 
other  survey  instrumentation. 

2.2.2  TEMTADS  2X2 

The  TEMTADS  2x2  is  a  man-portable  sensor,  however,  the  associated  components  (tablet,  heavy 
backpack,  and  batteries)  limit  the  ability  of  an  operator  to  collect  data  without  assistance  from 
additional  personnel.  For  this  demonstration,  two  personnel  were  utilized  during  the  data 
collection:  one  person  to  operate/navigate  the  sensor,  and  a  second  to  operate  the  tablet  and  place 
navigational  aids  and  monitor  the  tablet.  The  TEMTADS  2x2  sensor  is  also  not  ruggedized  to 
withstand  inclement  weather  (light  rain,  snow,  etc.).  This  proved  problematic  during  the  Andersen 
AFB  demonstration,  as  the  climate  on  Guam  is  hot  and  humid  with  frequent  precipitation. 

2.2.3  AGC  Data  Processing 

Compared  to  a  typical  EM61-MK2  survey,  the  AGC  dynamic  surveys  generated  more  files  that 
were  larger  in  size,  and  data  processing  was  time-consuming.  The  large  file  sets  can  be  difficult 
to  share  data  and  collaborate  with  non-local  colleagues.  The  increased  processing  time  made  it 
more  time  consuming  to  experiment  and  discover  better  ways  of  handling  the  data.  Weston 
processed  both  AGC  dynamic  and  cued  data;  however,  classification  of  dynamic  AGC  data  is 
not  an  option  within  the  commercially  available  UX- Analyze  extension  of  Geosoft  Oasis  montaj. 
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3.0  PERFORMANCE  OBJECTIVES 


Performance  objectives  developed  during  the  planning  phase  for  the  SWPG  and  Andersen  AFB 
demonstrations  are  provided  in  Table  3.1  and  Table  3.2,  respectively.  The  performance  objectives 
serve  as  a  basis  for  the  evaluation  of  the  performance  and  costs  of  the  demonstrated  technologies. 

Table  3.1.  Quantitative  Performance  Objectives  for  SWPG 


Performance 

Objectives 

Metric 

Success  Criteria 

Results 

Repeatability  of 
Instmment 
Verification  Strip 
(IVS)  measurements 

Amplitude  of 
electromagnetic 
(EM)  anomaly 
Measured  target 
locations 

Adv.  Sensors  Survey: 

Down-track  location  ±25  cm 

Adv.  Sensors  Cued: 

Library  match  >90%  using  three- 
criterion  metric  with  equal  weighting  to 
the  three  criteria  using  first  day’s  IVS 
inversion  as  the  library  item. 

Pass  -  All  IVS  events  achieved 
a  detection  offset  of  <25  cm  for 
all  seed  items 

Pass  -  All  IVS  events  achieved 
a  >90%  library  match  using  an 
equally  weighted  three-criterion 
match 

Complete  coverage 
of  the  demonstration 
site 

Footprint  coverage 
calculated  using 
UX-Process 

Footprint  Coverage 
Quality  Control 
(QC)  tool;  excludes 
inaccessible  areas. 

>85%  coverage  at  0.75-meter  (m)  line 
spacing;  and 

>98%  coverage  at  0.9-m  line  spacing 

Pass  -  99.8%  coverage  was 
achieved  at  a  0.75-m  line 
spacing 

Along-line 

measurement 

spacing 

Point-to-point 
spacing  from  data 
set 

98%  <15  cm  along-line  spacing 

Pass  -  99.6%  of  the  along-line 
spacing  was  <15  cm 

Detection  of  all  TOI 

Percent  detected  of 
TOI 

100%  of  TOI  detected  within  40-cm 
halo  of  the  surveyed  location 

Pass  -  100%  of  TOI  was 
detected  within  a  40-cm  halo 

Cued  interrogation 
of  anomalies 

Instillment  position 

100%  of  anomalies  where  the  center  of 
the  instmment  is  positioned  within  40 
cm  of  actual  target  location 

Fail  -  only  94%  of  the  cued 
measurements  were  within  the 
40-cm  metric 

Correctly  classify 

QC  seeds  and 
correctly  classify 
native  and 
population  seed 
items 

Percent  classified  as 
TOI 

100%  classified  as  TOI 

Pass  -  all  TOI  were  properly 
classified 

Correctly  identify 
group 

Percent  of  TOI  and 
excavated  non-TOI 
grouped  correctly 

85%  correctly  grouped  in  the  small, 
medium,  and  large  groups 

Pass  -  97%  were  assigned  to  the 
correct  group 

Correct  estimation 
of  extrinsic  target 
parameters 

Measured  location 
and  depth-to-center 
of  mass  of 
recovered  items 

X,  Y<15  cm  (1  o) 

Z  <10  cm  (1  o) 

Fail  -  only  67%  of  X,Y  <15  cm 
of  the  actual  measured  location 
Fail  -  only  75%  of  Z  <10  cm  of 
the  actual  depth 

Maximize  correct 
classification  of 
non-TOI 

Number  of  false 
alarms  eliminated 

Reduction  of  clutter  digs  by  >50% 
while  meeting  all  other  demonstration 
objectives 

Pass  -  84%  of  non-TOI  were 
correctly  classified 

Minimize  number  of 
anomalies  that 
cannot  be  analyzed 

Number  of 
anomalies  that  must 
be  classified  as 
“Unable  to 

Analyze” 

Reliable  target  parameters  can  be 
estimated  for  >95%  of  anomalies  on 
each  sensor  anomaly  list. 

Pass  -  only  2%  classified  as 
“Cannot  Analyze” 
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Table  3.2.  Quantitative  Performance  Objectives  for  Andersen  AFB 


Performance 

Objectives 

Metric 

Success  Criteria 

Results 

Repeatability  of 
IVS 

measurements 

Amplitude  of  EM 
anomaly 

Measured  target 
locations 

Adv.  Sensors  Survey: 

Down-track  location  ±25  cm 

Adv.  Sensors  Cued: 

Library  match  >90%  using  three- 
criterion  metric  with  equal  weighting  to 
the  three  criteria  using  first  day’s  IVS 
inversion  as  the  library  item. 

Fail  -  Two  IVS  events  exceeded 
the  detection  offset  of  <25  cm 

Fail  -  Only  87%  of  cued  IVS 
events  achieved  a  >90%  library 
match  using  an  equally 
weighted  three-criterion  match. 
Explanation  provided  in 
subsequent  sections. 

Complete 
coverage  of  the 
demonstration 
site 

Footprint 
coverage 
calculated  using 
UX-Process 
Footprint 

Coverage  QC 
tool;  excludes 
inaccessible 

areas. 

>  85%  coverage  at  0.50-m  line  spacing; 
and 

>  98%  coverage  at  0.60-m  line  spacing 

Pass  -  93.8%  coverage  was 
achieved  at  a  0.50-m  line 
spacing,  99.1%  at  0.60  m  line 
spacing 

Along-line 

measurement 

spacing 

Point-to-point 
spacing  from  data 
set 

98%  <25-cm  along-line  spacing 

Pass  -  100%  of  the  along-line 
spacing  was  <25  cm 

Detection  of  all 
TOI 

Percent  detected 
of  TOI 

100%  of  TOI  detected  within  40-cm 
halo  of  the  surveyed  location 

Pass  -  100%  of  TOI  was 
detected  within  a  40-cm  halo 

Cued 

interrogation  of 
anomalies 

Instmment 

position 

100%  of  anomalies  where  the  center  of 
the  instmment  is  positioned  within  40 
cm  of  actual  target  location 

Fail  -  Only  94%  of  the  cued 
measurements  were  within  the 
40-cm  metric. 

Explanation  provided  in 
subsequent  sections. 

Correctly  classify 
QC  seeds  and 
native  and 
population  seed 
items 

Percent  classified 
as  TOI 

100%  classified  as  TOI 

Pass  -  all  TOI  were  properly 
classified 

Correctly  identify 
group 

Percent  of  TOI 
and  excavated 
non-TOI  grouped 
correctly 

85%  correctly  grouped  in  the  small, 
medium,  and  large  groups 

Pass  -  98%  were  assigned  to  the 
correct  group 

Correct 
estimation  of 
extrinsic  target 
parameters 

Measured 
location  and 
depth-to-center  of 
mass  of 
recovered  items 

X,  Y  <15  cm  (la) 

Z  <10  cm  (la) 

Pass  -  a  of  X,Y  offsets  <15  cm 

Pass  -  a  of  Z  <10  cm  of  the 
actual  depth 

Explanation  provided  in 
subsequent  sections. 

Maximize  correct 
classification  of 
non-TOI 

Number  of  false 
alarms  eliminated 

Reduction  of  clutter  digs  by  >50% 
while  meeting  all  other  demonstration 
objectives 

Pass  -  81%  of  non-TOI  were 
correctly  classified 

Minimize 
number  of 
anomalies  that 
cannot  be 
analyzed 

Number  of 
anomalies  that 
must  be  classified 
as  “Unable  to 
Analyze” 

Reliable  target  parameters  can  be 
estimated  for  >95%  of  anomalies  on 
each  sensor’s  anomaly  list. 

Pass  -  only  2%  classified  as 
“Cannot  Analyze” 
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4.0 


TEST  DESIGN 


4.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

Overall,  each  Live  Site  Demonstration  was  conducted  with  three  primary  objectives: 

1.  Correctly  classify  all  TOIs. 

2.  Reduce  clutter  digs  by  at  least  50%. 

3.  Demonstrate  the  cost  and  performance  of  these  sensors  and  methods. 

In  general,  the  key  components  of  these  demonstrations  were: 

•  Site  preparation  (i.e.,  clearing  brush,  installing  seed  items,  installing  IVS). 

•  Dynamic  data  collection  with  advanced  sensors  (i.e.,  MetalMapper,  TEMTADS  2X2)  to 
detect  anomalies. 

•  Cued  data  collection  with  advanced  sensors  at  detected  anomaly  locations. 

•  Data  analysis  and  classification  to  estimate  extrinsic  parameters  (measured  location  and 
depths  of  items). 

•  Using  AGC  results  to  detennine  a  ranked  anomaly  list. 

•  Intrusive  investigation  of  selected  targets. 

Detailed  information  regarding  each  site’s  conceptual  experimental  design  is  contained  in  its 
respective  Final  Report.  A  summary  of  each  site’s  conceptual  experimental  design  is  found  in 
Table  4.1. 


Table  4.1.  Conceptual  Experimental  Designs 


Project 

Component 

Conceptual  Experimental  Design 

Southwestern  Proving  Ground 
(SWPG) 

Andersen  AFB 

Site  Preparation 

■  43  seed  items  installed 

■  Vegetation  thinned 

■  IVS  installation 

■  37  seed  items  installed 

■  Vegetation  thinned 

■  IVS  installation 

Data  Collection 

■  1 1 .23  acres  MetalMapper  dynamic  survey 

■  2,116  cued  surveys 

■  2.97  acres  TEMTADS  dynamic  survey 

■  1,195  cued  surveys 

Data  Processing 

■  Processed  dynamic  and  cued  data  with 
Geosoft  UX-Analyze 

■  Processed  dynamic  and  cued  data  with 
Geosoft  UX-Analyze 

Data  Analysis  and 
Classification 

■  Inverted  cued  results  for  classification 
using  library  matching  augmented  by 
visual  data  review 

■  Dig/No  Dig  list  produced  for 
demonstration  analysis 

■  Inverted  cued  results  for  classification 
using  library  matching  and  clustering 
augmented  by  visual  data  review 

■  Dig/No  Dig  list  produced  for 
demonstration  analysis 

Intrusive 

Investigation 

■  1,398  target  locations  intrusively 
investigated 

■  Each  item  was  photographed  and  attribute 
information  collected 

■  243  target  locations  intrusively 
investigated 
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4.2  SITE  PREPARATION 


Prior  to  AGC  data  collection,  Weston  established  sufficient  geodetic  control  points  at  each  site. 
Vegetation  removal  was  completed  when  necessary  and  permissible  at  each  site  to  allow  for 
quality  data  collection.  Weston  UXO  technicians  performed  the  surface  sweep  using  handheld 
analog  metal  detectors  (e.g.,  Schonstedt®  locators)  to  remove  surface  metal  and  any  explosive 
hazards  associated  with  potential  MECs.  IVS  and  blind  seeds  were  installed  at  each  site  in 
accordance  with  the  specifications  and  descriptions  contained  in  Geophysical  System  Verification 
(GSV):  A  Physics-Based  Alternative  to  Geophysical  Prove  Outs  for  Munitions  Response  (ESTCP 
2009).  Test  pits  were  established  near  the  IVS  at  a  quiet  location  free  of  subsurface  metal  and  were 
used  to  measure  the  signatures  of  TOI  expected  to  be  present  within  each  demonstration  area. 
Detailed  information  regarding  site-specific  preparation  is  contained  in  each  site’s  Final  Report. 

4.3  DATA  COLLECTION 

4.3.1  Southwestern  Proving  Ground  (SWPG),  Arkansas 

4.3. 1.1  Dynamic  Data  Collection 

Weston  perfonned  dynamic  detection  surveys  within  the  RF-15  demonstration  areas  April  18- 
29,  2013.  Dynamic  detection  data  were  collected  from  a  total  of  11.23  acres  using  the 
MetalMapper,  equating  to  an  average  of  1.4  acres  per  day.  Prior  to  the  survey,  a  map  of  projected 
survey  lines  in  RF-15  was  loaded  into  EM3DAcquire  to  use  for  navigation.  As  each  survey  line 
was  collected,  EM3DAcquire  displayed  a  colored  swath  the  width  of  the  sensor  footprint 
showing  the  operator  where  data  have  been  acquired.  Data  gaps  were  typically  identified  in  the 
field  and  re-collected  the  same  day. 

The  dynamic  detection  surveys  were  perfonned  with  the  MetalMapper  sensor  seated  within  a  sled, 
which  was  attached  to  the  front  mount-plate  of  a  diesel-powered  tele-lifter  (see  Figure  4.1).  The  tele- 
lifter  allowed  the  MetalMapper  to  be  raised  up  and  down  and  easily  maneuvered  side  to  side.  A 
monitor  mounted  within  the  vehicle  displayed  real-time  navigation  and  sensor  infonnation,  and 
allowed  the  operator  to  collect  data  in  both  dynamic  and  cued  survey  modes.  EM3DAcquire  was 
used  during  this  demonstration  to  control  data  acquisition  parameters,  storage  of  data,  and  navigation. 


Figure  4.1.  MetalMapper  at  SWPG 
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A  Trimble®  R8  Real  Time  Kinematic  (RTK)  GPS  was  used  for  navigation.  The  rover  head  was 
mounted  directly  over  the  center  of  the  MetalMapper  transmit  coil.  An  IMU  was  installed 
directly  below  the  rover  head  to  capture  pitch,  roll,  and  yaw  of  the  sensor. 

Dynamic  MetalMapper  survey  data  were  acquired  with  a  design  line  spacing  of  0.60  m.  Data 
were  initially  collected  with  a  0.75-m  line  spacing;  however,  it  was  observed  after  the  first  day  of 
dynamic  survey  that  a  0.75-m  spacing  was  resulting  in  data  gaps  because  of  the  soft  ground  and 
rutting  that  was  present  in  the  field,  so  a  tighter  line  spacing  of  0.6  m  was  used  to  achieve  full 
coverage. 

4.3. 1.2  Cued  Data  Collection 

Weston  performed  cued  data  collection  at  SWPG  May  13-30,  2013,  on  2,116  targets  that  were 
based  on  anomaly  lists  approved  by  ESTCP,  averaging  281  cued  locations  each  day  for  8  field 
days.  Cued  target  locations  were  loaded  into  EM3DAcquire,  which  was  used  for  navigation  as 
well  as  data  storage.  The  operator  positioned  the  MetalMapper  (Figure  4.1)  within  40  cm  of  the 
center  of  target  location  and  collected  a  cued  shot  for  a  60-second  period  over  the  anomaly.  To 
account  for  changing  soil  conditions,  background  shots  were  collected  once  an  hour  in  a  quiet 
area  identified  in  the  dynamic  data.  The  cued  data  were  reviewed  each  evening,  and  cued 
locations  that  fell  outside  the  40-cm  offset  metric  were  re-collected  as  necessary. 

4.3.2  Andersen  AFB,  Guam 
4.3.2. 1  Dynamic  Data  Collection 

Weston  performed  dynamic  detection  surveys  with  the  TEMTADS  2x2  using  EM3DAcquire  to 
control  data  acquisition  parameters,  storage  of  data,  and  real-time  monitoring  of  TEMTADS  2x2 
sensor  and  peripheral  IMU  and  RTK  data  streams.  A  Trimble  R8  RTK  GPS  was  used  for 
navigation.  The  rover  head  was  mounted  directly  over  the  center  of  the  TEMTADS  2x2  array. 
An  IMU  was  installed  directly  below  the  rover  head  to  capture  pitch,  roll,  and  yaw  of  the  sensor. 
Dynamic  TEMTADS  2x2  survey  data  were  acquired  with  a  design  line  spacing  of  0.50  m. 

Dynamic  detection  surveys  were  conducted  over  the  course  of  five  days  from  January  7-17, 
2014.  There  were  a  number  of  no-collection  days  during  this  timeframe  due  to  inclement  weather 
as  well  as  TEMTADS  2x2  sensor  and  computer  malfunctions.  Dynamic  detection  data  were 
collected  over  a  total  of  2.97  acres  using  the  TEMTADS  2x2,  equating  to  an  average  of  0.6  acres 
per  day  of  data  collection.  Data  gaps  were  typically  identified  in  the  field  and  re-collected  the 
same  day  to  ensure  full  coverage.  Dynamic  detection  survey  using  the  TEMTADS  2x2  is  shown 
in  Figure  4.2.  One  team  member  navigates  the  TEMTADS  2X2  while  the  other  member 
remotely  operates  the  computer  using  a  tablet  and  maintains  line  spacing  by  placing  visual 
markers  (i.e.  beanbags). 
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Figure  4.2.  TEMTADS  2x2  at  Andersen  AFB 


4.3. 2. 2  Cued  Data  Collection 

Weston  performed  cued  data  collection  over  the  course  of  10  days  between  January  18  and 
February  05,  2014,  at  1,195  anomaly  locations  that  were  based  on  the  approved  target  lists.  Cued 
data  collection  averaged  120  cued  locations  each  day  during  the  10  field  days.  Cued  target 
locations  were  reacquired  with  the  RTK  GPS  and  flagged  each  day  prior  to  data  collection.  The 
operator  then  positioned  the  TEMTADS  2x2  within  40  cm  of  the  center  of  the  flagged  location 
and  collected  a  cued  measurement  over  the  anomaly.  To  account  for  changing  background 
conditions,  background  measurements  were  collected  once  per  hour  in  a  quiet  area  identified  in 
the  dynamic  data  set.  The  cued  data  were  reviewed  each  evening,  and  cued  locations  that  fell 
outside  the  40-cm  offset  metric  were  re-collected  as  necessary. 
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5.0  DATA  ANALYSIS  AND  PRODUCTS 


5.1  DYNAMIC  DATA  PROCESSING 

The  raw  binary  AGC  .TEM  files  were  converted  to  American  Standard  Code  for  Information 
Interchange  (ASCII)  comma-separated  value  (.CSV)  files  using  data  conversion  software  (i.e., 
EM3D4  or  TEM2CSV).  Converted  data  were  then  imported  into  Geosoft  Oasis  montaj  for 
processing  and  analysis  using  scripted  import  routines.  Upon  import,  raw  data  were  inspected  to 
ensure  that  sensor  data  were  valid  and  that  peripheral  input  data  streams  (GPS,  IMU)  were 
present.  Dynamic  detection  data  were  imported,  processed,  and  validated  on  a  daily  basis. 

The  individual  sensors  within  the  AGC  dynamic  detection  data  were  then  located  and  exported  to 
a  separately  located  database,  with  each  sensor  assigned  a  unique  version  number  per  line  (e.g., 
Line  1,  sensor  1  equates  to  Line  1.1;  Line  1,  sensor  2  equates  to  Line  1.2).  Sensor  offsets  were 
calculated  in  reference  to  the  RTK  GPS  position  at  the  center  of  the  array,  with  IMU  data  used  to 
adjust  for  pitch,  roll,  and  yaw  in  the  sensor  array.  Data  analysis  and  anomaly  selection  were 
performed  on  the  z-axis  component  of  the  five  innermost  receiver  cubes  (cubes  2-6). 

The  dynamic  detection  data  were  then  levelled  using  a  de-median  background  removal  filter. 
Once  the  daily  data  had  been  imported,  validated,  and  levelled,  the  data  were  then  merged  into  a 
master  site  database  containing  all  dynamic  data  collected  to  date.  Once  data  collection  was 
complete,  the  master  database  was  used  for  gridding,  anomaly  selection,  and  analysis. 

5.2  ANOMALY  SELECTION 

Anomalies  were  selected  from  processed  advanced  dynamic  detection  data  using  the  Geosoft 
Blakely  grid  peak  detection  algorithm.  To  determine  a  suitable  anomaly  selection  threshold, 
dynamic  test  data  were  acquired  over  a  20-millimeter  (mm)  projectile  buried  at  the  target  detection 
depth  of  15  cm.  An  example  of  the  20-mm  test  strip  data  collected  is  shown  in  Figure  5.1. 
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Background  noise  analyses  were  performed  on  anomaly-free  locations  within  the  test  strip  data. 
A  target  conservative  threshold  of  7  times  the  root  mean  square  (RMS)  noise  level  was  chosen  to 
allow  the  detection  of  a  20mm  projectile  at  a  depth  of  15  cm  below  ground  surface. 

5.3  CUED  DATA  PROCESSING 

Cued  data  processing  was  perfonned  using  the  UX-Analyze  Advanced  extension  in  Geosoft 
Oasis  montaj.  Cued  background  data  were  imported  and  qualitatively  verified,  with  any  outliers 
removed  from  the  background  dataset.  After  background  data  had  been  verified,  cued  anomaly 
data  were  imported,  verified  for  completeness,  and  background  corrected  using  the  cued 
background  data  spatially  and  temporally  closest  to  the  cued  anomaly  location. 

Inversions  were  performed  on  each  cued  anomaly  using  both  single-source  and  multi-source 
models  to  extract  target  parameters,  fit  coherence,  and  predict  locations  and  depths  for  each 
model.  The  primary  parameters  used  for  classification  were  the  three  polarizabilities  ((31,  (32,  and 
(33)  calculated  for  each  single-source  and  multi-source  modeled  result. 

Daily  quality  control  (QC)  was  performed  on  the  cued  anomaly  data  in  which  the  cued  location 
(RTK  GPS  location),  modeled  locations,  and  flagged  locations  were  compared  to  verify  that  the 
center  of  the  AGC  sensor  array  was  within  the  40-cm  radius  of  the  anomaly  source.  Targets 
outside  the  40-cm  metric  were  identified  and  re-collected  as  necessary. 

After  the  individual  inverted  locations  from  the  cued  sensors  were  complete,  the  data  were 
combined  into  a  Master  Dig  List  for  each  anomaly  group.  These  Master  Dig  Lists  contained  one 
entry  for  each  predicted  anomaly  from  the  inversions  of  the  cued  data. 
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6.0  PERFORMANCE  ASSESSMENT 


This  section  discusses  the  performance  assessments  for  AGC  sensors  at  each  of  the  sites.  A 
summary  of  the  performance  objectives  and  results  for  each  demonstration  project  are  located  in 
Section  3,  Table  3.1  (SWPG)  and  Table  3.2  (Andersen  AFB)  of  this  report.  Detailed  information 
regarding  each  site’s  performance  assessment  results  is  contained  in  its  respective  Final  Report. 

6.1  REPEATABILITY  OF  IYS  MEASUREMENTS 

This  objective  involved  the  repeatability  of  the  detection  location  and  classification  (cued  data) 
of  seed  items  in  dynamic  IVS  data  collection.  Seed  item  offsets  for  each  dynamic  IVS  data 
collection  event,  and  library  match  statistics  for  each  cued  IVS  data  collection,  were  tracked 
throughout  the  life  of  the  dynamic  portion  of  the  project.  This  objective  for  dynamic  collection 
was  considered  to  be  met  if  all  locations  of  seed  items  as  detected  in  the  IVS  data  were  offset 
<25  cm  from  the  actual  surveyed  location.  This  objective  for  cued  collection  was  considered  to 
be  met  if  the  library  match  statistic  for  all  seed  items  cued  in  the  IVS  was  >90%  when  using  a 
three-criterion  metric  with  equal  weighting  to  the  three  criteria  when  measured  against  the  first 
day’s  cued  IVS.  These  perfonnance  objectives  for  both  dynamic  and  cued  data  were  met  at 
SWPG. 

At  Anderson  AFB,  the  dynamic  metric  of  25  cm  was  exceeded  on  two  occasions,  therefore,  the 
dynamic  metric  was  not  met.  The  cued  metric  was  not  met  because  not  all  of  the  library  match 
statistics  were  >90%  during  the  duration  of  the  cued  survey.  The  exact  source  of  the  failures 
were  not  detennined,  however  a  contributing  factor  was  most  likely  the  IVS  construction  or  the 
seed  items  installed  in  the  IVS.  Due  to  limited  suitable  space  available  to  install  an  IVS  within 
the  North  Ramp  Parking  (NRP)  area,  the  existing  IVS  established  for  the  EM61-MK2  survey 
being  perfonned  in  support  of  the  clearance  activities  was  used  for  the  demonstration.  Schedule 
40  Industry  Standard  Objects  (ISOs)  were  used  in  the  IVS,  which  may  have  been  a  contributing 
factor  to  the  inconsistent  results  of  the  deeper  small  ISO  seed  item  IVS03.  It  is  unclear  what 
caused  the  match  statistic  deviations  over  the  medium  ISO  seed  items.  To  assess  data  usability, 
production  data  were  evaluated  from  any  day  that  IVS  deviations  were  observed,  and  no 
indications  of  a  system  failure  with  respect  to  data  usability  were  observed.  Seed  items  cued 
during  the  days  in  question  were  properly  classified  with  high  confidence  statistical  matches,  and 
no  deficiencies  were  noted  in  sensor  performance. 

6.2  COMPLETE  COVERAGE  OF  THE  DEMONSTRATION  SITE 

This  objective  measured  the  effectiveness  of  the  dynamic  detection  survey  as  a  function  of  the 
amount  of  coverage  of  the  demonstration  area  by  the  AGC  sensor.  This  objective  was  considered 
to  be  met  if  the  dynamic  detection  survey  achieved  its  instrument-specific  objective  when 
analyzed  using  the  UX-Process  Footprint  Coverage  QC  tool. 

At  SWPG,  the  UX-Process  Footprint  Coverage  QC  tool  was  used  to  analyze  the  georeferenced 
positions  of  the  center  of  the  AGC  sensor  array.  Data  were  collected  at  a  0.50-m  line  spacing  to 
eliminate  gaps  caused  by  ruts  and  rough  terrain.  This  objective  was  met  because  99.8%  of  the 
site  was  covered  at  a  0.75-m  line  spacing. 
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At  Andersen  AFB,  the  UX-Process  Footprint  Coverage  QC  tool  was  used  to  analyze  the 
georeferenced  positions  of  the  center  of  the  TEMTADS  2x2  sensor  array.  Data  were  collected  at 
a  0.50  m  line  spacing  to  eliminate  gaps  caused  by  ruts  and  rough  terrain.  This  objective  was  met, 
as  93.8%  of  the  site  was  covered  at  a  0.50  m  line  spacing,  and  99.1%  at  0.60  m  line  spacing. 

6.3  ALONG-LINE  MEASUREMENTS 

This  objective  evaluated  the  along-line  data  density,  or  sample  separation,  of  the  TEMTADS  2x2 
dynamic  detection  dataset  acquired  within  the  NRP  area.  The  metric  for  this  objective  was  the 
point-to-point  distance  as  measured  using  the  UX-Process  Sample  Separation  utility. 

At  SWPG,  this  objective  was  considered  to  be  met  if  98%  of  the  data  had  an  along-line  spacing 
of  15  cm  or  less.  The  UX-Process  Sample  Separation  tool  was  used  to  analyze  the  along-line 
spacing  of  the  georeferenced  data  positions  of  the  MetalMapper  sensor  array.  This  objective  was 
met  because  99.6%  of  the  data  had  a  sample  separation  of  15  cm  or  less. 

At  Andersen  AFB,  this  objective  was  considered  to  be  met  if  98%  of  the  data  had  an  along-line 
spacing  of  25  cm  or  less.  The  UX-Process  Sample  Separation  tool  was  used  to  analyze  the  along- 
line  spacing  of  the  georeferenced  data  positions  of  the  TEMTADS  2x2  sensor  array.  This 
objective  was  met,  because  100%  of  the  data  had  a  sample  separation  of  25  cm  or  less. 

6.4  DETECTION  OF  ALL  TOI 

This  objective  evaluated  the  dynamic  detection  capabilities  of  the  AGC  sensors.  The  metric  for 
this  objective  was  considered  to  be  met  if  100%  of  native  and  non-native  TOI  were  detected 
within  a  40-cm  halo  of  their  recorded  locations.  Non-native  TOI  within  these  demonstration 
areas  included  a  combination  of  blind  seed  items,  which  were  completely  blind  to  the  data 
collection  and  processing  teams,  as  well  as  QC  seed  items  for  which  the  locations  were  known. 

At  SWPG,  this  objective  was  met  because  all  TOI  were  successfully  detected  within  40  cm  of 
the  recorded  locations.  TOI  included  43  seed  items  (non-native  TOI)  installed  by  Weston  prior  to 
the  dynamic  detection  survey,  as  well  as  1  MEC  item  (native  TOI).  The  average  seed  time  offset 
for  this  investigation  was  10.25  cm. 

At  Andersen  AFB,  this  objective  was  met  because  all  TOI  were  successfully  detected  within  the 
TEMTADS  2x2  dataset  within  40  cm  of  the  recorded  locations.  TOI  included  19  seed  items 
(non-native  TOI)  installed  by  Weston  prior  to  the  dynamic  detection  survey.  The  average  seed 
item  offset  for  this  investigation  was  10.2  cm. 

6.5  CUED  INTERROGATION  OF  ANOMALIES 

This  objective  evaluated  the  positioning  of  the  instrument  during  data  collection  in  relation  to  the 
actual  anomaly  location.  The  metric  for  this  objective  was  considered  to  be  met  if  the  center  of 
the  instrument  was  positioned  within  40  cm  of  the  actual  anomaly  location  for  100%  of  the  cued 
anomalies.  To  evaluate  this  objective,  the  offset  between  the  center  of  the  AGC  sensor  array  and 
the  surveyed  location  of  each  recovered  item  was  calculated. 
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At  SWPG,  this  objective  was  not  achieved  because  only  94%  of  the  cued  measurements  were 
within  the  40-cm  offset.  Of  the  1,398  cued  measurements  that  were  intrusively  investigated, 
1,301  were  within  the  40-cm  offset  metric,  and  86  were  outside  of  40  cm.  The  remaining  11 
reacquired  anomalies  were  determined  to  be  no-contacts  and  were  removed  from  this  evaluation. 

At  Andersen  AFB,  this  objective  was  not  achieved  because  only  94%  of  the  cued  measurements 
were  within  the  40-cm  offset.  Direct  comparison  of  TEMTADS  2x2  cued  locations  to  the  actual 
location  of  recovered  items  is  not  possible.  Detailed  intrusive  infonnation  and  locations  of 
recovered  items  were  not  collected  by  the  contractor  perfonning  the  MEC  removal  activities.  As  a 
result,  this  objective  was  evaluated  by  comparing  the  offset  between  the  center  of  the  TEMTADS 
2x2  array  and  the  fit  location  of  each  source.  Of  the  1,195  cued  measurements  that  were  analyzed, 
1,127  were  within  the  40-cm  offset  metric,  and  68  were  outside  of  40  cm.  Of  the  68  cued 
measurements  that  were  outside  of  40  cm,  39  of  these  exceedances  were  from  the  977  targets  (4% 
of  TEMTADS  targets)  selected  from  dynamic  TEMTADS  2x2  data,  and  29  were  from  the  218 
targets  (13%  of  the  EM61  targets)  selected  from  EM61-MK2  data.  The  increased  exceedance  rate 
at  targets  selected  from  the  lower  resolution  EM61-MK2  dynamic  detection  survey  suggests  that 
the  data  from  which  the  target  locations  are  selected  will  influence  failure  rates. 

6.6  CORRECTLY  CLASSIFY  QC  SEEDS 

This  objective  evaluated  the  effectiveness  of  the  advanced  classification  process  to  properly 
classify  TOI  present  within  the  survey  area.  The  objective  was  considered  to  be  met  if  100%  of 
the  QC  seeds,  population  seeds,  and  native  TOI  were  placed  on  the  TOI  list. 

At  SWPG,  a  ranked  anomaly  list  was  submitted  to  the  ESTCP  Program  Office  for  evaluation. 
This  objective  was  met,  because  all  TOI  were  properly  classified  as  Category  1  digs  (likely  TOI). 
This  listing  included  one  105mm  projectile  (MEC)  recovered  in  Anomaly  Group  1. 

At  Andersen  AFB,  a  ranked  anomaly  list  was  submitted  to  the  ESTCP  Program  Office  for 
evaluation.  This  objective  was  met,  because  all  TOI  were  properly  classified  as  Category  1  digs 
(likely  TOI). 

6.7  CORRECTLY  IDENTIFY  GROUPS 

This  objective  evaluated  the  effectiveness  of  the  advanced  classification  process  to  properly 
assign  each  excavated  TOI  and  non-TOI  into  the  small,  medium,  or  large  grouping.  The 
objective  was  considered  to  be  met  if  85%  of  the  anomalies  placed  on  the  dig  list  were  properly 
grouped.  Dig  results  for  the  ranked  anomaly  list  submitted  to  the  ESTCP  Program  Office  were 
analyzed  to  verify  size  groupings. 

At  SWPG,  of  the  195  anomalies  placed  in  the  dig  list,  190  were  assigned  to  the  correct  size 
group,  and  5  were  assigned  an  incorrect  group.  This  objective  was  met  because  97%  of  the 
anomalies  were  correctly  classified. 

At  Andersen  AFB,  intrusive  investigations  were  performed  to  the  specifications  of  the  third- 
party  contractor’s  work  plan,  and  did  not  follow  the  standard  ESTCP  intrusive  investigation 
requirements,  so  dig  data  obtained  did  not  include  precise  locations,  depths,  photos,  or 
descriptions  of  items  recovered.  Since  precise  data  were  not  available,  a  qualitative  analysis 
was  perfonned  on  the  dig  results  to  determine  if  the  proper  group  was  assigned  to  each  item. 


15 


Of  the  220  anomalies  placed  on  the  dig  list,  186  were  assigned  to  the  correct  size  group,  3  were 
assigned  an  incorrect  group,  and  31  did  not  have  adequate  data  to  make  a  comparison.  This 
objective  was  met,  as  98%  of  the  anomalies  that  had  adequate  dig  data  for  a  qualitative 
comparison  were  correctly  classified. 

6.8  CORRECT  ESTIMATION  OF  EXTRINSIC  TARGET  PARAMETERS 

This  objective  evaluated  the  accuracy  of  the  target  parameters  that  were  estimated  during  the 
data  inversion  process  by  comparing  the  predicted  extrinsic  target  parameters  to  the  measured 
results  recorded  during  the  intrusive  investigation.  This  objective  was  considered  to  be  met  if  the 
estimated  X  and  Y  locations  were  within  15  cm  and  the  estimated  depths  were  within  10  cm. 

At  SWPG,  this  objective  was  not  met  because  only  67%  of  the  predicted  locations  were  within 
15  cm  of  the  actual  measured  location,  and  only  75%  of  the  predicted  depths  were  within  10  cm 
of  the  actual  depths.  The  high  percentage  of  item  locations  that  were  not  predicted  correctly  was 
most  likely  a  result  of  the  quantity  of  frag  that  was  encountered  within  the  survey  area. 
Approximately  97%  of  the  digs  resulted  in  small  pieces  of  frag  that  were  either  too  numerous  or 
too  small  to  model  well,  thus  yielding  poor  fit  locations. 

At  Andersen  AFB,  detailed  locations  and  depths  of  items  recovered  were  not  recorded  during  the 
intrusive  investigation,  so  a  complete  evaluation  of  all  cued  data  could  not  be  performed. 
Therefore,  this  objective  was  evaluated  by  comparing  the  fit  locations  derived  from  the  cued 
TEMTADS  data  to  the  RTK  surveyed  locations  and  depths  of  the  seed  items  installed  within  the 
TEMTADS  demonstration  area.  Based  on  that  assessment,  this  objective  was  met,  as  the 
standard  deviation  for  each  of  the  X  and  Y  horizontal  offset  and  the  Z  vertical  offset  were  <15 
cm  and  <10  cm,  respectively. 

6.9  MAXIMIZE  CORRECT  CLASSIFICATION  OF  NON-TOIs 

This  objective  concerns  the  component  of  the  classification  problem  that  involves  false  alarm 
reduction.  The  metric  for  this  objective  is  the  number  of  cued  anomalies  that  can  be  correctly 
classified  as  non-TOI.  The  objective  was  considered  to  be  met  if  we  were  able  to  reduce  clutter 
digs  by  >50%  while  meeting  all  other  demonstration  objectives.  Dig  results  for  the  initial  ranked 
anomaly  list  submitted  to  the  ESTCP  Program  Office  were  used  to  assess  the  number  of  non-TOI 
that  were  correctly  classified. 

At  SWPG,  this  objective  was  met  because  84%  of  non-TOI  were  correctly  classified.  In  this 
classification  scenario,  100%  of  TOI  were  correctly  classified  while  achieving  a  false  positive 
rate  of  only  15%. 

At  Andersen  AFB,  this  objective  was  met  because  81%  of  non-TOI  were  correctly  classified.  In 
this  classification  scenario,  100%  of  TOI  were  correctly  classified  while  achieving  a  false 
positive  rate  of  only  18%. 
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6.10  MINIMIZE  NUMBER  OF  ANOMALIES  THAT  CANNOT  BE  ANALYZED 


This  objective  evaluated  how  well  the  modeled  results  of  the  inversion  process  correlated  to  the 
observed  data.  A  fit  coherence  metric  was  calculated  for  each  model  during  data  inversion,  and 
was  used  as  the  basis  for  determining  whether  reliable  parameters  could  be  estimated  from  the 
data.  Modeled  results  with  a  fit  coherence  of  <0.8  were  placed  in  the  ‘cannot  analyze’  category. 
The  objective  was  considered  to  be  met  if  reliable  parameters  could  be  estimated  for  >95%  of  the 
anomalies  on  each  sensor  anomaly  list. 

At  SWPG,  this  objective  was  met  because  98%  of  the  cued  data  collected  inverted  with  a  fit 
coherence  >0.8. 

At  Andersen  AFB,  this  objective  was  met  because  98%  of  the  cued  data  collected  inverted  with  a 
fit  coherence  >0.8. 
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7.0  COST  BENEFIT  ANALYSIS 


This  cost  assessment  for  the  SWPG  and  Andersen  AFB  demonstrations  includes  a  summary  list 
of  the  project  costs  and  potential  savings  from  the  classification  process. 

7.1  COST  SUMMARY 

Tables  7.1  and  7.2  present  a  cost  summary  for  the  technology  used  at  each  demonstration  site  for 
this  project. 


Table  7.1.  Details  of  Project  Costs,  SWPG 


Phase  of  Work 

Elements  of  Work 

Estimated  Costs 

Site  Setup 

Site  prep,  surface  sweep,  seeding,  IVS 
installation 

$81,109 

Dynamic  Detection  Survey 

Includes  effort  for  field  data  collection  and 
processing/anomaly  selection 

$52,241 

Cued  Survey 

Equipment 

$13,500 

Cued  data  collection 

$34,583 

Processing  and  classification 

$22,730 

Total  cost  per  target  for  cued  survey 

$3 3. 46/target  for  2,1 16  targets 

Intrusive  Investigation 

Intrusive  investigation  of  1,398  anomalies, 
reacquire,  demo  operations,  and  related  costs 

$195,860 

Total  cost  per  target  to  intrusively  investigate 

$140. 10/target  for  1,398  targets 

Table  7.2.  Details  of  Project  Costs,  Andersen  AFB 


Phase  of  Work 

Elements  of  Work 

Estimated  Costs 

Mobilization/Demobilization 

Travel  for  a  geophysicist  and  UXO 
technician  and  shipment  of  TEMTADS  2x2 
and  other  equipment 

$16,000 

Site  Setup 

Site  prep,  seeding,  IVS  installation, 
weather  delays 

$15,000 

Dynamic  Detection  Survey 

Includes  effort  for  field  data  collection 
(2.97  acres),  equipment,  expenses,  labor  for 
geophysicists  and  UXO  technicians, 
processing/anomaly  selection,  weather 
delays/downtime  and  local  subcontractor 
support  (10  days) 

$34,000 

Total  cost  per  acre  for  dynamic  survey 

$  11,447/acre 

Cued  Survey 

Equipment,  data  collection,  labor  for 
geophysicists  and  UXO  technicians  and 
expenses,  weather  delays/downtime,  and 
local  subcontractor  support  (5  days) 

$51,000 

Data  processing  and  classification 

$34,000 

Total  cost  per  anomaly  for  cued  survey 

$71/anomaly  for  1,195  anomalies 
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7.2  COST  DRIVERS 


This  analysis  presents  and  compares  the  costs  of  the  dynamic  and  cued  surveys  to  the  outcome  of 
the  classification  process. 

During  the  SWPG  demonstration,  2,1 16  targets  were  surveyed  in  cued  mode  with  MetalMapper. 
Based  on  the  information  listed  in  Table  7.1,  the  cost  for  the  cued  survey  and  classification  was 
$33.46  per  target.  Of  these  2,116,  a  total  of  1,398  were  intrusively  investigated  at  a  cost  of 
$140.10  per  target. 

During  the  Andersen  AFB  demonstration,  a  total  of  2.97  acres  of  dynamic  detection  survey  data 
was  collected  using  TEMTADS  2x2  over  the  course  of  10  days.  The  total  cost  for  the  dynamic 
detection  survey  was  approximately  $34,000  equating  to  $1 1,447  per  acre.  The  cued  survey,  data 
analysis,  and  classification  at  1,195  anomalies  totaled  $85,000  equating  to  $71  per  anomaly. 

Due  to  logistical  considerations  with  the  location  of  Guam,  travel  (mobilization/demobilization) 
as  well  as  site  setup  are  separately  reported  in  Table  7.2.  Weston  also  procured  the  help  from  a 
local  subcontractor  to  provide  additional  UXO  technician  escort  and  avoidance  support,  vehicles, 
storage,  and  equipment.  TEMTADS  2x2  shipping  was  approximately  $12,000.  Expenses  and 
some  labor  were  expended  during  weather  delays  and  downtime  due  to  equipment  issues. 

Weston  considers  the  costs  incurred  for  each  AGC  demonstration  and  intrusive  investigation  an 
overestimation  of  the  actual  costs  that  would  be  necessary  for  future  projects  because  these  costs 
included  equipment  delivery  delays,  significant  logistical  considerations,  and  lengthy  setup  and 
orientation  processes.  In  addition,  these  AGC  demonstration  sites  required  additional  efforts. 

7.3  COST  BENEFIT 

During  the  SWPG  demonstration,  cued  surveys  and  processing  were  perfonned  for  $33.46  per 
target,  and  intrusive  investigation  was  perfonned  for  $140.10  per  target.  The  classification 
process  eliminated  83%  of  anomalies  from  the  intrusive  investigation,  yielding  a  potential  cost 
savings  on  this  project  of  $1 14,898.00,  based  on  the  following  factors: 

•  1,398  anomalies  at  $140.10/anomaly  for  intrusive  investigation  equals  approximately 
$195,860. 

•  Reduction  of  1,154  anomalies  equals  a  reduction  of  $161,675  in  excavation  costs. 

•  MetalMapper  cost  for  cued  survey  and  classification  of  1,398  anomalies  at  $33.46/ 
anomaly  equals  a  cost  of  $46,777. 

•  Total  cost  savings  under  this  scenario  equals  $  1 14,898. 

During  the  Andersen  AFB  demonstration,  cued  surveys  and  processing  were  performed  for 
$71.00  per  target,  and  intrusive  investigation  was  perfonned  for  $150.00  per  target.  The 
classification  process  eliminated  81%  of  anomalies  from  the  intrusive  investigation,  yielding  a 
potential  cost  savings  on  this  project  of  $57,955,  based  on  the  following  factors: 

•  1,195  anomalies  at  $150.00/anomaly  for  intrusive  investigation  equals  approximately 
$179,250. 
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•  Reduction  of  952  anomalies  equals  a  reduction  of  $  142,800  in  excavation  costs. 

•  TEMTADS  2X2  cost  for  cued  survey  and  classification  of  1,195  anomalies  at  $71.00/ 
anomaly  equals  a  cost  of  $84,845. 

•  Total  potential  cost  saving  under  this  scenario  equals  $57,955. 
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8.0  IMPLEMENTATION  ISSUES 


Several  implementation  issues  arose  during  the  AGC  demonstrations,  such  as  equipment 
shipping  logistics,  software  malfunctions,  broken  cables  caused  by  cattle,  and  site  accessibility 
issues  because  of  heavy  rains  causing  flooded  site  conditions.  These  types  of  setbacks  are  typical 
of  any  site  and  should  be  expected  when  planning  field  operations.  Weston  discussed  suggested 
improvements  to  the  system  with  Geometries  once  survey  operations  were  completed,  to  include 
the  following: 

•  Simplify  the  cables  -  One  of  the  more  time-consuming  aspects  of  dealing  with 
MetalMapper  is  cable  management.  Each  of  the  receivers  and  transmitters  needs  to  be 
individually  attached  and  secured.  Weston  suggested  that  the  cables  be  reduced  to  a 
single  “transmitter”  cable  and  a  single  “receiver”  cable. 

•  Weatherproof  the  system  -  Limited  protection  of  cables  and  electronics  is  built  into 
the  AGC  systems.  On  the  MetalMapper,  the  computer  sits  facing  up,  with  no  shield  to 
protect  it  from  rain/debris/dust.  Most  of  the  cables  do  not  have  any  additional  sheathing 
beyond  the  manufacturing  minimum,  making  them  prone  to  pinches  and  pulls.  On  the 
TEMTADS  system,  the  clamshell  is  not  sealed  from  outside  moisture  and  dust.  Sensor 
failures,  likely  due  to  moisture  entering  the  clamshell,  resulted  in  several  days  of 
downtime  and  system  troubleshooting.  A  rubber  gasket  to  seal  the  top  and  bottom 
clamshell  pieces,  as  well  as  the  cable  harness  entering  the  clamshell,  is  a  possible 
solution. 

•  Structural  improvement  -  Several  components  of  the  MetalMapper  require  structural 
improvements.  The  GPS  needs  to  be  redesigned  for  additional  stability  and  less 
maintenance  (replacing  screws/nuts  that  are  consistently  coming  loose).  The  cushions 
used  for  vibration  dampening  need  to  better  adhere  to  the  MetalMapper  or  be  integrated 
into  the  instrument  design.  The  basket  that  holds  the  computer  should  be  adjusted  in  size 
so  that  it  actually  holds  the  computer.  Currently,  a  combination  of  duct  tape  and  come- 
along  straps  are  used  to  secure  the  computer  in  place  on  the  sled. 

•  Wheels  -  Currently  the  TEMTADS  wheels  are  held  on  by  zip  ties,  as  opposed  to  nylon 
bolts  or  cotter  pins.  These  zip  ties  failed  on  almost  a  daily  basis,  leading  to  downtime  to 
reattach  wheels  and  re-collect  affected  lines. 

•  Tow  versus  push  sled  design  -  The  MetalMapper  sled  is  currently  constructed  to  be 
“pushed”  during  survey.  For  the  cued  investigation,  this  setup  is  ideal  because  it  allows 
the  user  greater  control  and  visibility  when  moving  the  sensor  into  a  specific  location.  For 
the  dynamic  survey,  a  towed  sled  might  be  a  better  option,  depending  on  the  terrain.  The 
installation  of  a  ball-hitch  on  the  sled  would  allow  the  user  to  have  a  choice  in  the  field 
and  potentially  increase  the  versatility  of  the  instrument. 
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•  Other  modifications  -  Other  modifications  discussed  included  a  touchscreen  interface 
for  data  collection,  altering  the  color  schemes  of  the  software  for  greater  visibility,  and 
altering  data-file  nomenclature  for  easier  data  management.  In  addition,  a  modification  of 
EM3DAcquire  is  suggested  to  allow  for  display  of  dynamic  data  collection  progress  (plot 
tracks  on  map,  similar  to  MetalMapper  EM3D  interface)  and  the  option  to  import  a 
flag/target  list  into  TEMDatalogger  to  avoid  operator  input  errors. 
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APPENDIX  A  POINTS  OF  CONTACT 


Point  of 
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Phone 
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4800  Mark  Center  Drive 

Suite  17D08 

Alexandria,  VA  22350-3605 

571-372-6400  (V) 
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Director,  SERDP 
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